1. Field of the Invention
The present invention relates to electronic digital imaging, more particularly, to a digital imaging integrated circuit with integral image buffer and print controller.
2. The Prior Art
Digital imaging is the process of acquiring a digital representation of a visual signal and manipulating the representation for a desired result. The representation is acquired using an array of photosensitive pixels. The analog output of each pixel is read, converted to digital form, and stored in local memory, called an image buffer. The digital data from the image buffer is manipulated to produce the final image and then stored and/or displayed. Two characteristics of the digital image directly affect the size of the image buffer: the number of pixels and the resolution of the analog-to-digital conversion. The number of pixels for inexpensive photographic imaging is typically about 300,000, from a 640xc3x97480 pixel array. The digital resolution of each pixels in photographic imaging is typically between 8 and 12 bits. Thus, for the typical color photo image, the image buffer needed to hold the raw data from the photodetector array is about 300 to 450 kilobytes (KB).
Some digital imaging systems, such as electronic cameras, give the user the ability to directly print a hard copy of an image. Currently, this means that most systems have a standard printer port, which can be a serial or parallel port using an appropriate protocol and data format. The port is used to connect to a standalone printer with a standard interface. The internal processor of the system reads data from the image buffer, manipulates it to generate the appropriate data format for the printer, and then transfers data from memory to the printer interface, which, in turn, transmits the data to the printer.
A typical prior art system is shown in FIG. 2. Note that the image controller 100 and the image buffer 104 are two independent devices connected by external circuitry. The printer interface 116 has a standardized output that sends data to a standalone printer 110. The main reason for having independent devices like this is historical: the technologies are different enough that companies do not take a integrated system design approach to the creation of imaging systems. More specifically, imaging companies do not design memory devices, memory companies do not design imaging devices, and neither of them design print controllers. The technologies are different enough that companies do not think to try to produce both, and particularly do not think to integrate them into a single device. Consequently, when designing systems for image processing, one had to choose an image processor, add to it general purpose memory devices, and tack on a printer interface.
One additional issue regarding the printer interface is power. There are several basic types of print engines. It is currently not practical to build a laser print engine into a hand-held camera because of their extremely large power requirements. It is also not practical to build an ink-jet print engine into a hand-held camera because it cannot be made small enough, particularly when considering that it needs ink reservoirs. Nor are ink-jet printers immune to changes in orientation, a detriment in any kind of hand-held equipment.
For these reasons and others, thermal print engines are the most practical for designing into hand-held electronic cameras. The main shortcoming of thermal print engines is that, when the heating elements of a thermal printer are activated, the power surge is relatively large. This can cause problems with the camera batteries. Because a hand held camera is small, the batteries are also small and not generally capable of supply large surges of power, at least for very long.
In the system of FIG. 2, image data from the image sensor 102 must pass through the image controller 100 to get to the image buffer 104. Meanwhile, with the Von Neumann architecture shown, the image processor 112 is also performing memory accesses to its program memory 106, causing bus collisions and potentially reducing the speed at which data can be transferred from the image sensor 102 to the image buffer 104. To minimize such problems, the image controller 100 incorporates a high-speed cache, typically in the form of a dual port first-in-first-out (FIFO) memory 122. Image data accumulates in the FIFO 122 as it is received from the image sensor 102, and is transferred to the image buffer 104 as bus time becomes available.
The same bus collision problems occur when reading image data from the image buffer 104. Some processes require data on a periodic basis, for example, the driver 114 for the liquid crystal display (LCD) 108 and/or the printer interface 116. Again, in order to guarantee that data will be available when needed, data is read ahead out of the image buffer 104 into a FIFO 124 within the image controller 100, and the data is taken from the FIFO 124, which is updated as memory access cycles become available.
Thus, moving image data to and from the image buffer requires an extra step, that of needing a cache to make sure that image data is available when needed. Obviously, the more hardware there is in a system, the more complicated the system becomes, and potentially, the less potential there is for operating at faster speeds. Thus, there continues to be a need for an imaging processor that eliminates that extra caching step required by imaging system of the prior art.
An object of the present invention is to provide a combined image processor and image buffer on a single microcircuit device.
Another object is to provide an integrated image processor in which memory transfers to and from the image buffer are independent of the image processor central processing unit and prioritized by need.
The basis of the image processor microcircuit (IP) of the present invention is that the combination of an image processing microcircuit with an external image buffer operates too slowly for most practical uses unless the image processor has an internal means for caching data from the image buffer. And the addition of an on-board cache greatly complicates the design and operation of such an image processor. The present invention integrates the image buffer with the typical image processor functions described above, in effect avoiding the speed problems inherent with an external buffer and the complications associated with an internal cache.
The main components of the IP of the present invention are a central processing unit (CPU) with program memory, an image buffer (IB), an image buffer access prioritizer (IBAP), an image sensor interface (ISI), and a host interface. Optionally, there are a liquid crystal display interface (LCDI), a print controller (PC), and a general purpose communications interface (GPCI). The CPU provides general processing functions, and has separate program and data buses. The host interface provides communications between the IP and any master device. The GPC interface provides communications between the IP and any other external device.
The preferred image buffer is a synchronous random-access memory device (SRAM) having a minimum size of Pxc3x97N, where P is number of pixels in the image sensor used with the IP and N is the number of bits for each pixel. Access to the IB is controlled by the IBAP, which determines which internal device is to be given access to the IB at any given time. The internal devices, in order of priority, include the LCDI, ISI, PC, and CPU. All input control signals from these devices to the IB, particularly the read and write enable signals, pass through the IBAP. The IBAP outputs prioritization signals to the various devices, in the form of a BUSY signal. When a device senses that its BUSY signal is asserted (active), it delays any accesses to the IB that are not currently in progress. The IBAP asserts the BUSY signals of lower priority devices after sensing that a device wants access to the IB. After the higher-priority device has completed access, the BUSY signals are deasserted.
The ISI reads image data from the IS and writes it to the IB. The particular IS of this implementation provides a 12-bit digital value to represent the analog value from a pixel. Pixels are read from the IS one line at a time. The CPU sets up the IS and ISI before each line, including the IB address where the line of image data is to be stored and the IB address where reference data can be found. As each pixel of data is read from the IS, the ISI has the ability to add or subtract a reference value from the IB to the pixel data and store the modified pixel data in the IB. Typically, this capability is used to subtract pixel noise data as the image is being read.
After the IS and ISI are set up, the CPU initiates the line read. In the present implementation, data is transferred from the IS at 20% of the ISI internal clock rate. Thus, there are 5 internal ISI cycles for each pixel data transfer. In cycle 1, the pixel data and reference data are read. In cycle 2, the reference data is added or subtracted from the pixel data, if the calculation is enabled. In cycle 3, the result is written to the IB. If the pixel is that last of the line, the ISI generates an interrupt to the CPU. If it is not the last line of the image, the CPU sets up the next line and continues.
Note that only 3 of the 5 internal cycles are used by the ISI, which means that two cycles are available for the LCDI, PC, and CPU to access the IB.
The IB is also used as temporary storage for the LCD image data, which is generated by the CPU. The LCDI controls the LCD and transfers data directly from the IB to LCD without going through the CPU. The CPU sets the IB address where the LCD data is located, then starts the LCDI. The LCDI starts reading at the address and, when complete, starts over again, until halted by the CPU. As currently implemented, the LCDI needs to access the IB about once every 3 pixel data transfers, and uses one of the two extra ISI cycles to do so.
The IB is also used as temporary storage for the print image data, which is generated by the CPU. The present invention uses a thermal print engine that has a row of heating elements, one for each pixel of an image row. An image pixel is generated on the paper by heating a spot for a period of time, the length of which determines how dark the spot is. The current implementation allocates 8 bits of data for each pixel of the print image. This means that the heating element may be activated for a heating period that is from 0 to {fraction (255/256)}ths of the maximum heating time, in increments of {fraction (1/256)}. The increment is denoted a time slice. Typically, the maximum heating time can range from about 1 ms to about 100 ms.
The heating elements are driven by latches, one for each heating element, and the latches set from a shift register that has a location for each latch.
The print engine is controlled by the PC. The process by which the PC determines how long a heating element is to remain on is an iterative process. Each pixel data is compared 256 times to a threshold value that starts at 0 and increments after each comparison. If the pixel data is greater than the threshold value, the corresponding latch is set high. Otherwise it is set low.
In implementation, each pixel of a line is compared to the same threshold value and the result is clocked into a shift register that has a position for each pixel of the line. After all pixels of the line have been compared to the same threshold value, the shift register is clocked into the latches and the output of the latches is strobed to turn on the heating elements. The length of the strobe is the same as a time slice. During the time slice, the same line of pixels is compared to the threshold value that has been incremented by one. This continues for all 256 increments of the threshold value. After the line is complete, the CPU instructs the print engine to advance the paper one line, and the process starts again with the next line of pixels.
One aspect of the present invention is the power-saving manner in which the heating elements are activated. The heating elements require a relatively large surge of power when first applied, which is a detrimental to the small batteries typically used in an electronic camera. Consequently, the line of pixels is divided into smaller sets. Each set goes through the complete 256-level threshold comparison before the next set. Thus, only a portion of the heating elements are activated at one time, greatly reducing the power surge on the batteries.